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object  (a  cube)  was  affected  as  this  orientation  was  varied 
relative  to  that  of  a  fixed  control  (a  cubical  response 
box).  In  the  second,  we  studied  how  accurately  the  observer 
can  perceive  and  project  a  trajectory  presented  within  the 
display,  as  a  function  of  the  trajectory's  orientation.  .In 
the  third  phase,  we  used  a  real-time  control  task  to  measure 
directly  the  relative  utility  of  the  three  dimensions  of  the 
display. 
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Introduction 

Three-dimensional  displays  have  reached  the  stage  of 
development  at  which  they  can  be  seriously  considered  for  a 
variety  of  applications.  However,  since  the  technology  is  new, 
not  very  much  is  known  about  the  human-factors  aspects  of  such 
displays:  how  accurately  the  operator  can  perceive  various 
properties  of  objects  presented  in  the  display,  and  how  the 
presence  of  the  third  dimension  enriches  both  perception  and 
performance  of  tasks  monitored  with  the  aid  of  the  display.  Bolt 
Beranek  and  Newman  (BBN)  had  previously  developed  a  prototype 
vibrating  mirror  3-D  display,  called  SpaceGraph.  The  operator 
views  a  CRT  screen  reflected  in  the  mirror,  and  when  the  mirror 
is  vibrated,  the  virtual  image  of  the  CRT  face  sweeps  through  a 
volume  behind  the  mirror.  True  3-D  images  can  be  drawn  as  arrays 
of  points,  with  the  X  and  Y  coordinates  of  each  point  being 
specified  by  where  on  the  CRT  face  a  point  is  instantaneously 
displayed,  and  the  Z,  or  depth  coordinate  is  specified  by  the 
instant  in  the  mirror  cycle  at  which  the  point  is  displayed.  As 
a  result,  the  virtual  image  viewed  by  the  observer  is  truly  three 
dimensional,  and  therefore  undergoes  perspective  transformations 
when  the  observer  moves  his  head,  exactly  as  if  it  were  a  real 
object.  The  mission  of  the  work  performed  under  this  contract 
was  to  explore  some  perceptual  and  display-control  compatibility 
issues  in  the  context  of  the  SpaceGraph  display. 


BBN  Report  Mo  5584 


Bolt  Beranek  and  Newman  Inc 


The  work  was  organized  into  three  phases.  In  the  £irstr  we 
studied  how  the  speed  and  accuracy  of  the  operator's  decisions 
about  the  orientation  of  a  displayed  object  (a  cube)  was  affected 
as  this  orientation  was  varied  relative  to  that  of  a  fixed 
control  (a  cubical  response  box) .  In  the  second,  we  studied  how 
accurately  the  observer  can  perceive  and  project  a  trajectory 
presented  within  the  display,  as  a  function  of  the  trajectory's 
orientation.  In  the  third  phase,  we  used  a  real-time  control 
task  to  measure  directly  the  relative  utility  of  the  three 
dimensions  of  the  display. 

BHASE  1:  fal  Effects  of  Orientation  (Report  No.  4724) 

in  the  first  phase,  we  studied  the  effects  of  differences  of 
orientation  on  a  spatial  identification  task.  The  observer  was 
presented  on  each  trial  with  an  outline  cube  in  the 
three-dimensional  display.  The  bottom  face  of  the  stimulus  cube 
was  marked  with  an  orientation  cue  that  the  operator  used  to 
identify  the  cube's  orientation.  One  of  the  other  five  faces  was 
also  marked,  with  a  "stimulus  key"  consisting  of  two  concentric 
circles,  and  the  observer's  task  was  to  identify  which  of  the 
five  faces  was  so  marked  (top,  far,  near,  left,  or  right),  and 
press  the  corresponding  response  key  on  a  physical  response  cube. 

A  first,  general  finding  was  that  the  depth  dimension  of  the 
display  was  less  salient,  or  at  least  took  longer  to  extract, 
even  in  a  true  3-D  display,  than  the  lateral  or  vertical 
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dimensions.  The  difference  in  decision  times  was  larger  for 
pairs  of  stimuli  that  differed  mainly  in  depth  than  for  pairs 
that  differed  mainly  in  lateral  or  vertical  position.  However, 
responses  to  stimuli  at  the  back  of  the  display  volume  were  made 
just  as  rapidly  as  stimuli  at  the  front. 

We  found  three  different  types  of  functions  relating 
decision  time  to  orientation  of  the  stimulus  cube:  (a)  flat 
functions  in  which  the  decision  time  was  short  and  independent  of 
stimulus  cube  orientation,  and  (b)  two  types  of  curvilinear 
function  in  which  decision  time  increased  and  decreased 
systematically  with  orientation.  The  curvilinear  functions 
appeared  to  be  made  up  of  two  distinct  parts:  one  involving  a 
"plateau"  shaped  function  with  skirts;  and  the  other  consisting 
of  a  "peak"  shaped  function,  with  the  peak  involving  long 
reaction  times,  and  associated  with  orientations  in  which  one  or 
more  of  the  axes  in  the  display  was  reversed  relative  to  the 
corresponding  axis  of . the  control. 

Each  of  the  three  types  of  functions  (flat,  plateau-shaped, 
and  peak-shaped)  we  associated  with  the  observer's  use  of  a 
particular  strategy  for  selecting  a  response.  The  flat  functions 
arose  when  the  observer  was  able  to  determine  the  response 
uniquely  from  the  spatial  position  of  the  stimulus  button  with 
respect  to  the  world  coordinates  of  the  display  volume.  Thus, 
when  the  stimulus  cube  was  rotated  about  its  vertical  Y-axis,  the 
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top  stimulus  button  was  invar iantly  located  at  the  top  of  the 
displayed  cube  for  all  orientations  of  the  stimulus  cube,  because 
the  rotation  axis  passed  through  the  stimulus  button.  As  a 
result,  the  observed  decision  times  for  the  top  response  were 
unaffected  by  stimulus  cube  orientation  for  this  task.  The 
functions  for  the  four  other  responses  were  strikingly  different. 
Those  for  the  front  and  back  responses  were  plateau  shaped, 
whereas  those  for  the  left  and  right  responses  were  predominantly 
peak-shaped. 

The  next  three  experiments  produced  related  results.  These 
experiments  involved  rotations  of  the  stimulus  cube  about  (a)  the 
depth  or  Z-axis,  (b)  the  lateral  X-axis,  and  (c)  either  the  X-  or 
the  Y-  or  the  Z-axis.  When  rotation  was  about  only  the  lateral, 
X-axis  of  the  cube,  the  spatial  strategy  was  applied  to  the  left 
and  right  stimulus  buttons,  consonant  with  their  invariant 
position  within  the  display.  When  rotations  about  any  one  of  the 
three  axes  might  be  presented  on  any  trial,  no  use  of  the  spatial 
strategy  was  possible,  since  none  of  the  stimulus  keys  remained 
in  a  constant  position.  Peak-shaped  functions  were  associated 
with  the  left  and  right  responses  in  three  of  the  four 
experiments;  the  exception  involved  rotation  about  the  lateral, 
X-axis  /  in  which  case  the  superior  spatial  strategy  could  be 
applied. 
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We  hypothesized  that  the  difference  in  performance  between 
the  left  and  right  responses,  on  the  one  hand,  and  the  near  and 
far  responses,  on  the  other,  was  related  to  the  symmetry  of  the 
cue  we  had  used  for  indicating  the  orientation  of  the  stimulus 
cube:  a  capital  letter  V  drawn  on  the  cube's  bottom  face,  with 
its  apex  touching  the  front  edge.  The  responses  that  were 
asymmetric  with  respect  to  this  cue  (near  and  far  faces)  yielded 
plateau-shaped  functions,  whereas  those  that  were  symmetric  (left 
and  right  faces)  yielded  peak-shaped  functions.  Thus  for  the 
former  pair,  a  relational  strategy  could  be  used,  in  which  the 
spatial  relationship  between  the  stimulus  key  and  the  asymmetric 
aspect  of  the  orientation  cue  indicated  the  correct  response 
(e.g.  the  apex  of  the  V  "pointed"  toward  the  near  face) .  For  the 
remaining  pair  of  responses,  the  observer  was  forced  to  adopt  a 
much  less  efficient  strategy  for  selecting  the  response,  a 
rotational  strategy.  This  involved  the  observer  mentally 
rotating  into  an  orientation  from  which  the  cube  appeared  "head 
on"  (i.e.  the  apex  of  the  V  pointed  toward  the  observer),  and 
then  making  a  decision  according  to  whether  the  stimulus  button 
appeared  on  the  left  or  right. 

Thus  the  difference  between  the  substantial  display  control 
incompatibility  found  for  the  left  and  right  responses  and  the 
much  smaller  incompatibility  found  for  the  near  and  far  responses 
appeared  to  be  due  to  the  cue  chosen  for  indicating  the 
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orientation  of  the  stimulus  cube.  If  so,  it  should  be  possible 
to  reduce  the  incompatibility  by  using  a  more  appropriate 
orientation  cue. 

ibl. -Effects  Of  Cue  Symmetry  (Report  No.  5101) 

Our  next  three  experiments  tested  the  conclusions  from  the 
foregoing  analysis.  In  the  first,  we  again  used  a  cue  that  was 
symmetric  with  respect  to  one  dimension  and  asymmetric  with 
respect  to  the  other  (a  capital  letter  A) ,  but  we  positioned  the 
cue  so  as  to  interchange  the  responses  associated  with  the 
symmetric  and  asymmetric  aspects  of  the  cue.  That  is,  the  left 
and  right  responses  were  associated  with  the  asymmetric  dimension 
of  the  cue,  and  the  near  and  far  responses  were  associated  with 
the  symmetric  dimension.  The  results  supported  our  analysis,  in 
that  the  near  and  far  responses  now  showed  major  display-control 
incompatibility  effects.  Their  reaction  time  functions  were 
sharply  peaked,  and  the  left  and  right  responses  showed  much 
flatter  plateau-shaped  functions  appropriate  to  use  of  the 
relational  strategy. 

The  second  and  third  experiments  used  an  orientation  cue 
that  was  asymmetric  in  bfifh  dimensions:  a  capital  letter  V  on  the 
bottom  face  of  the  cube,  as  before,  but  made  asymmetric  by  the 
addition  of  a  serif  to  the  left  upright  of  the  V  pointing  toward 
the  left  face  of  the  cube.  The  addition  of  this  serif 
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significantly  reduced  the  display-control  incompatibility  of  both 
pairs  of  responses.  The  peak-shaped  functions  disappeared 
because  observers  were  now  able  to  use  a  relational  strategy  for 
all  responses;  and  the  responses  associated  with  this  strategy 
also  became  shorter  as  a  result  of  all  responses  being  selectable 
from  only  two  instead  of  three  strategies. 

In  a  final  experiment  in  the  series ,  we  examined  the  effects 
of  prolonged  practice  on  performance,  by  repeating  the  initial 
experiment  exactly.  The  relative  shapes  of  the  functions  had 
changed  very  little,  although  responses  were  significantly  faster 
throughout,  and  the  size  of  the  peaks  associated  with  the  left 
and  right  responses  was  greatly  reduced,  due  mainly  to  greatly 
reduced  scatter  in  the  response  times  resulting  from  elimination 
of  the  extremely  long  response  times  found  in  the  first 
experiment. 

PHASE  2:  Perception  of  Direction  (Report  No.  5582) 

A  prime  application  area  for  3-D  displays  could  be  in  air 
traffic  control.  The  possibility  of  integrating  the 
plan-position  and  the  vertical-situation  indicator  displays  into 
a  single  display  might  be  one  of  the  more  obvious  goals.  Before 
using  such  a  combined  display,  it  will  be  necessary  to  find  out 
how  accurately  the  operator  can  estimate  and  project  directions 
within  the  display.  Such  projection  would  be  needed  for  deciding 
whether  a  collision  was  likely  to  occur. 
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In  the  task  that  we  developed,  the  observer  was  presented  on 
each  trial  with  an  outline  cube,  of  side  about  18  cm,  with  a 
vector  of  3  cm  pointing  from  the  cube's  center  toward  one  of  the 
five  sides  of  the  cube  away  from  the  observer.  The  observer's 
task  was  to  place  a  point,  using  a  graphical  tablet  as  an  input 
device,  on  the  cube's  surface  where  the  vector  would  pierce  it  if 
extended  (the  "piercing"  point).  The  cube  was  composed  of  dotted 
lines  along  each  of  its  twelve  edges,  and  its  center  was  marked 
by  a  small  cluster  of  points.  The  vector  originated  in  this 
cluster.  The  tablet  was  marked  with  an  outline  representing  the 
five  faces  of  the  cube  unfolded  onto  a  flat  surface,  with  the 
back  face  of  the  cube  in  the  middle.  As  the  observer  moved  the 
stylus  within  the  outline,  the  point  representing  the  piercing 
point  moved  correspondingly  on  the  surface  of  the  cube  in  the 
display. 

There  were  80  vectors,  16  pointed  toward  each  of  the  five 
faces,  generated  by  applying  the  rules  of  symmetry  to  only  two 
different  vectors.  Each  observer  made  20  responses  for  each  of 
the  80  vectors.  Analyses  of  the  data,  performed  separately  for 
each  observer,  showed  three  principal  effects  that  produced  error 
in  the  piercing  points.  These  effects  appeared  to  be  independent 
of  one  another,  and  to  be  superimposed  in  the  data. 

The  first  effect  was  a  strong  bias  for  the  distribution  of 
piercing  points  to  fall  along  the  linear  extension  of  the  vector 


i  § 


»l 


*Vl 


Wl 


4 


v| 

g 


...  ^ 
«►  v 

5'  ••'* 


|£ 


BBN  Report  No  5584 


Bolt  Beranek  and  Newman  Inc. 


as  projected  from  the  observer's  viewpoint  upon  the  inner  surface 
of  the  cube.  The  variance  of  the  response  distributions  was 
always  large  in  the  direction  of  the  projection,  relative  to  that 
in  the  orthogonal  direction.  For  vectors  pointing  at  the  top, 
bottom,  or  sides  of  the  cube,  the  projections  onto  those  faces 
were  a  set  of  lines  parallel  to  the  depth  axis,  so  that 
variability  in  depth  was  much  greater  than  variability  in  the 
frontal  plane. 

The  second  effect  was  a  systematic  error  in  piercing  point 
away  from  the  true  target  point  toward  the  closest  principal  axis 
of  the  space  (the  vertical,  lateral,  and  depth  axes). 
Furthermore,  the  amount  of  angular  error  grew  in  proportion  to 
the  angular  deviation  of  the  target  point  from  the  closest 
principal  axis.  In  effect,  our  observers  tended  to  perceive 
oblique  vectors  as  more  nearly  horizontal  or  vertical  than  they 
really  were. 

The  third  effect  was  a  distortion  of  the  perception  of  depth 
as  a  function  of  merioian  in  the  frontal  plane.  In  effect,  the 
frontal  plane  was  perceived  as  slanted  in  depth,  with  the  top  of 
the  plane  tilted  back  relative  to  the  bottom.  The  amount  and 
direction  of  the  slant  differed  significantly  across  observers. 
We  believe  that  this  particular  distortion  may  arise  from  a 
combination  of  two  effects:  (1)  the  known  backwards  slant  of  the 
vertical  horopter,  and  (2)  rotation  of  the  frontal  plane  about  a 
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vertical  axis  due  to  mild,  uncorrected  astigmatism  in  one  or  both 
eyes  of  our  observers.  We  informally  measured  the  axis  and 
degree  of  astigmatism  in  each  eye  of  our  observers.  The  observed 
perceptual  distortions  are  in  general  agreement  for  each  observer 
with  predictions  based  on  the  measured  astigmatism. 


PHASE  3;  Perception  and  Control  of  Motion  (Report  No.  5583) 

Several  of  the  possible  applications  for  a  3-D  display  such 
as  spaceGraph  could  involve  the  real-time  control  of  images 
presented  on  the  display.  Our  earlier  experiments,  and  other 


informal  observations  made  under  a  variety  of  conditions, 
suggested  that  the  depth  dimension  of  the  displayed  images  or 
their  position  was  perceived  less  quickly  and  less  accurately 
than  the  lateral  or  vertical  dimensions.  The  third  phase  of  work 
was  performed  to  quantify  these  differences,  and  to  determine 
their  causes. 

The  task  selected  was  a  closed-loop  tracking  task  with  a 
first-order  unstable  controlled  element,  and  with  a  sum-of-sines 
disturbance  function  added  to  the  loop  input  to  perturb  the 
system  away  from  its  rest  state.  This  task  was  implemented  for 
one-,  two-,  and  three-axis  tracking,  although  only  the  one-  and 


two-axis  tasks  were  used  in  the  current  work.  Performance  was 
compared  on  five  separate  tracking-tasks.  Three  of  these  were 
single-axis  tasks,  one  for  each  of  the  major  axes  of  the  display 
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(lateral  or  X,  vertical  or  Y,  and  depth  or  Z).  The  remaining  two 
tasks  were  two-axis  tasks ,  in  which  the  controlled  element  moved 
in  a  plane  containing  the  lateral  and  the  vertical  axes  (XY) ,  or 
the  lateral  and  the  depth  axes  (XZ) .  In  all  tasksr  the 
controlled  element  was  a  single  point.  The  display  volume  was 
outlined  by  a  cube  about  18  cm  on  the  side,  and  an  irregular 
array  o£  dimmer  points  appeared  behind  the  back  face  of  the  cube 
to  provide  a  textured  background  as  a  reference  for  the  moving 
point.  The  center  of  the  cube,  corresponding  to  the  rest  state 
of  the  controlled  element,  was  marked  by  two  points  offset 
slightly  from  the  center  along  the  main  diagonal.  The  control 
was  a  two-axis  force  stick  ("stiff"  stick).  During  preliminary 
studies,  the  difficulty  of  the  task  was  specified  by  varying  the 
frequency  (time  constant)  of  the  pole  of  the  instability,  so  that 
it  was  just  possible  to  maintain  control  in  the  most  difficult  of 
the  five  tasks.  .After  some  preliminary  familiarization,  the 
observers  were  trained  concurrently  on  all  five  tasks  during 
daily  one-hour  sessions.  Experimental  data  were  collected  on  the 
eighth  and  ninth  sessions.  The  data  in  the  ninth  session  were 
collected  to  test  the  need  for  the  textured  background.  The 
background  array  was  removed  for  this  session,  so  the  stable 
referents  in  the  display,  along  with  the  controlled  point,  were 
the  outline  cube  and  the  two  points  marking  its  center. 


p 


BBN  Report  Bo  5584 


Bolt  Beranek  and  Newman  Inc. 


Performance  on  the  lateral  X  and  vertical  Y  axes  were  very 
similar,  as  expected.  Secondly,  performance  on  each  axis  in  the 
two-axis  XY  task  and  on  the  X  axis  in  the  XZ  task  was  very 
similar  to  those  on  the  component  single-axis  tasks;  and  in 
agreement  with  earlier  findings  that  two  independent  tracking 
tasks  can  be  performed  concurrently  without  loss,  if  the  display 
and  control  are  appropriately  integrated.  The  most  interesting 
comparisons  were  made  between  the  single-axis  X  and  Y  tasks,  on 
the  one  hand,  and  the  single-axis  z-task,  on  the  other. 
Performance  was  significantly  worse  on  the  depth  axis,  with  the 
tracking  score  (standard  deviation  of  the  position  of  the 
controlled  point)  about  60%  larger  than  that  found  for  either  the 
lateral  or  vertical  axes. 

Inspection  of  plots  of  the  operator's  gain,  phase,  and 
remnant  as  a  function  of  frequency  suggested  that  there  was 
little  difference  between  the  phase  functions  for  the  depth  and 
frontal  axes,  but  that  observers  reduced  their  gain  substantially 
when  tracking  in  the  depth  dimension,  and  remnant  power  was 
appreciably  higher  at  low  frequencies.  The  experimental  data 
were  modeled  \  iteratively  modifying  the  parameters  of  the 
optimal  contro  mi  1  of  the  human  operator,  separately  for  each 
axis.  Comparison  ^  the  parameters  for  the  lateral  and  vertical 
(i.e.  frontal)  axes,  on  the  one  hand,  and  the  depth  axis,  on  the 
other,  showed  that  observation  noise  levels  were  about  4  dB 
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higher  for  the  depth  axis,  and  in  addition  the  operator  delay  was 
increased  substantially  (by  about  100-150  ms) .  This  result  was 
confirmed  informally  by  repeating  the  model  runs  with  the  delay 
parameters  fixed,  and  finding  that  this  produced  a  substantially 
worse  fit  to  the  data.  A  possible  conclusion  is  that  more 
processing  was  required  to  extract  the  position  of  the  target 
when  tracking  in  the  depth  dimension  than  when  tracking  laterally 
or  vertically,  and  in  addition  it  was  harder  to  determine  the 
position  of  the  controlled  object  in  the  depth  axis  tasks.  An 
alternative  explanation  that  cannot  be  ruled  out  is  that  the 
operator  was  not  operating  linearly  in  the  depth  dimension,  but 
rather  made  no  response  until  some  threshold  was  exceeded.  Since 
linear  operation  is  required  by  the  optimal  control  model,  this 
would  cast  doubt  on  the  model  results. 

IMPLICATIONS  FOR  FURTHER  RESEARCH 
(a)  Orientation 

In  our  present  research,  we  found  that  the  time  required  to 
perceive  the  orientation  of  a  displayed  object  is  shorter  when  an 
observer  is  able  to  use  a  relational  rather  than  a  rotational 
strategy.  This  requires  that  the  object  be  asymmetric  along  each 
of  its  principal  axes.  These  results  were  obtained  using  static 
views  of  an  object  at  particular  orientations.  Further  work  is 
needed  to  determine  how  rapidly  and  accurately  an  observer  can 
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perceive  and  follow  the  changing  orientation  of  a  dynamically 
tumbling  and  translating  object.  Such  results  would  be  important 
in  applications  involving  dynamic  displays,  such  as  docking  with 
a  tumbling  platform  or  tracking  a  feature  on  a  tumbling  object. 

(b)  Direction 

Many  practical  applications  of  3-D  displays  will  require 
that  observers  be  able  to  accurately  perceive  direction  within 
the  space.  We  observed  two  major  types  of  systematic  error  when 
observers  predicted  the  point  at  which  a  vector  would  pierce  the 
surface  of  an  enclosing  cube.  Further  research  is  needed  to 
determine  the  bases  of  these  perceptual  distortions,  and  the 
means  for  minimizing  or  eliminating  their  effects  in 
applications. 

One  observed  distortion  was  the  displacement  of  perceived 
piercing  points  away  from  the  true  target  point  towards  the 
closest  principal  axis  (vertical,  lateral,  or  depth)  of  the 
space.  One  interpretation  suggests  that  we  tend  to  perceive  a 
vector  as  more  nearly  vertical  or  horizontal  than  it  is  truly. 
However,  because  of  the  presence  of  visible  edges  in  the 
displayed  cube,  it  is  also  possible  that  the  observed 
displacement  represents  a  sensory  or  perceptual  repulsion  of  the 
perceived  piercing  points  away  from  the  visible  edges.  Further 
research  is  needed  to  separate  these  two  confounded  explanations. 
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This  might,  for  example,  include  an  experiment  in  which  the 
visible  cube  edges  are  removed  so  that  the  observer-controlled 
point  moves  about  on  an  invisible  surface. 

The  second  observed  distortion  was,  in  effect,  an  oblique 
slant  of  the  frontal  plane,  different  for  each  observer.  We  have 
suggested  that  this  distortion  is  due  to  a  combination  of  several 
perceptual  effects,  one  arising  from  mild  uncorrected  astigmatism 
in  one  or  both  eyes  of  our  observers.  Further  experimental  work 
is  needed  to  test  this  explanation,  possibly  including 
experiments  in  which  perceptual  effects  are  measured  after 
inducing  controlled  amounts  of  astigmatism  along  known  meridians. 

(c)  Motion 

The  experiments  on  tracking  in  a  3-D  display  only  scratch 
the  surface  of  possible  research  in  this  area.  More  work  is 
needed  to  determine  whether  the  increased  processing  delay  found 
when  tracking  in  the  depth  dimension  is  real,  and  if  so,  whether 
it  is  possible  to  modify  the  display  so  as  to  reduce  its  effects. 
The  controlled  object  consisted  of  only  a  single  point,  with  the 
result  that  stereo-optical  disparity  was  the  only  cue  to  its 
depth.  If  a  larger  object  were  controlled,  its  changes  in 
subtended  retinal  angle  would  offer  additional  cues.  A  second 
area  where  more  research  is  needed  is  on  extending  the  tracking 
task  to  three  dimensions,  where  the  controlled  object  is 
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controlled  in  all  three  dimensions  simultaneously,  and  to  six 
dimensions,  where  the  attitude  of  the  controlled  object  is  under 
control  as  well  as  its  position.  These  tasks  are  of  interest 
because  they  approach  real-world  applications,  and  also  because 
they  can  be  used  to  answer  questions  about  the  appropriateness  of 
various  types  of  controls. 

A  further  area  where  research  is  needed  is  the  control  of 
3-D  objects  when  the  coordinate  axes  of  the  control  mechanism 
does  not  correspond  to  those  of  the  displayed  object. 

(d)  Control  Mechanisms 

In  the  first  set  of  experiments,  the  communication  between 
the  operator  and  the  display  was  in  one  direction  only;  the 
display  showed  an  image,  and  the  operator  viewed  that  image  and 
selected  a  response.  The  response  had  no  effect  on  the  display 
other  than  to  begin  a  new  trial.  The  "control"  device  with  which 
the  response  was  made  had  the  same  shape  as  the  displayed  object 
(a  cube)  because  in  these  experiments  we  were  more  interested  in 
the  effects  of  mismatches  of  orientation  between  the  displayed 
image  and  the  control  than  in-  the  effects  of  less  optimal 
mappings  from  the  display  onto  the  control  device.  There  is 
little  reason  to  doubt  that  the  findings  from  the  standard 
human-factors  literature  apply,  i.e.,  responding  becomes 
progressively  more  difficult,  slower,  and  less  abstract,  as  the 
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relationship  between  display  and  control  device  moves  from  strict 
isomorphism,  to  a  less-spatial  mapping,  and  then  to  a 
more-abstract  mapping.  The  significant  performance  improvement 
obtained  in  the  experiments  on  orientation  when  the  control  and 
displayed  image  were  in  corresponding  orientations  suggested 
that,  where  the  gravitational  vertical  in  the  displayed  image 
always  corresponded  to  the  vertical  axis  of  the  display  device, 
it  would  be  useful  to  give  the  operator  a  control  device 
consisting  of  a  horizontal  wheel  with  which  to  rotate  the 
orientation  of  the  displayed  image  to  an  optimal  position. 
Further  research  is  needed  to  test  this  suggestion. 

A  similar  control  device  could  be  useful  in  a  refinement  of 
the  set  of  experiments  on  the  perception  of  direction.  Here,  the 
highest  accuracy  in  projecting  the  vector  within  the  display 
volume  occurred  when  the  observer's  viewpoint  was  most  nearly 
aligned  with  the  vector  (i.e.  it  was  nearly  possible  to  sight 
down  it).  To  allow  the  operator  to  sight  down  any  vector,  the 
control  device  needs  two  control  axes.  A  track-ball  might  be  a 
good  way  to  accomplish  this;  and  the  top  of  the  ball  could  be 
rolled  in  the  direction  that  the  vertical  axis  should  be  tipped. 
Another  option  would  be  to  use  a  joy-stick  connected  so  that  a 
departure  from  its  (spring-loaded)  vertical  orientation  would 
cause  the  vertical  axis  of  the  display  volume  to  incline  such 
that  it  remained  parallel  with  the  joystick. 
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In  the  second  and  third  sets  o£  experiments,  the  loop  from 
control  device  to  display  was  closed,  so  that  movements  of  the 
control  device  affected  the  contents  of  the  display.  Operators 
had  no  difficulty  in  understanding  the  mapping  of  the  control 
device  onto  the  resulting  effects  that  appeared  on  the  display. 
Use  of  the  tablet,  which  is  a  two-dimensional  input  device,  was 
appropriate  in  the  experiment  on  trajectory  projection,  because 
we  wished  the  controlled  point  to  move  only  on  a  surface.  The 
fact  that  operators  were  able  to  use  the  forward-backward 
direction  of  movement  of  the  force  stick  to  control  either  the 
vertical  or  the  depthwise  movement  of  the  controlled  element  in 
the  tracking  task  suggested  that  a  three-dimensional  force  stick 
would  be  an  appropriate  input  device  for  a  three-dimensional 
control  task. 
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